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ABSTRACT 


Engineering educators call for a widespread implementation of hybrid learning to respond to 
rapidly changing demands of the 21st century [1]. In response to this call, a junior-level course in 
the Construction Engineering program entitled Construction Equipment and Heavy Construction 
Methods was converted into a hybrid learning model. The overarching goal in the hybrid course 
development was to take the content that can be engaged outside the class to an online platform 
so that class time can be used more efficiently for authentic and realistic open-ended problems 
included in-class and homework assignments. This study reports the design, development, and 
evaluation of this hybrid course and provides practical implications for hybrid course development. 

Key words: hybrid learning, evaluation, construction engineering 


INTRODUCTION 

Hybrid learning, interchangeably referred to as blended learning, in its most generic sense, is an 
educational approach that combines online instruction with face-to-face instruction [2, 3]. Because 
of its blending feature, the hybrid format is also considered as the best of two worlds providing the 
benefits of online and face-to-face environments [3]. In this particular study, hybrid learning refers 
to a combination of online and face-to-face instruction by reducing the physical meeting times. In 
hybrid learning, lectures can be replaced by interactive activities. Teaching assistants can facilitate 
these interactive activities under the supervision of the instructor. The instructor then has more 
time to interact with individual students and enhance the quality of the course through sustained 
course development and innovation [2, 4]. 
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When executed well, hybrid learning provides several benefits at the student, faculty, and 
institutional level. Researchers argued that blended learning courses were in high demand because 
they increase convenience and flexibility, improve learning outcomes, and improve the efficiency of 
classroom use [5]. A recent meta-analysis of online education by the U.S. Department of Education 
also revealed that students in online learning conditions performed modestly better than students 
in face-to-face conditions, and students in hybrid learning conditions performed better than both 
completely online and face-to-face conditions [6]. Similarly, Dziuban et al. found that blended learn¬ 
ing courses produced comparable or superior success rates compared to face-to-face or fully online 
modes when college and gender contributions were removed [7]. 

Prior research indicates that professors choose blended or hybrid learning for three main reasons: 
1) improved pedagogy, 2) increased access/flexibility, and 3) increased cost effectiveness [8, 9]. 
Hybrid learning approaches increase the opportunities for learner-centered pedagogies such as 
active learning strategies and group work [10]. Learner flexibility and convenience is also of growing 
importance as more nontraditional students, who have job and family commitments, seek additional 
education [8, 11]. In addition, hybrid learning holds the potential to increase the quality in higher 
education by “taking faculty out of the easily automated business of delivery of information and 
allowing them to refocus their attention with students on critical thinking skills” [1]. 

Engineering educators call for a widespread implementation of hybrid learning to respond 
to rapidly changing demands of the 21st century [1]. Many higher education institutions seem to 
have already started implementing hybrid learning. For example, the College of Engineering at 
the University of Wisconsin-Madison adopted a hybrid learning model as an innovative solution 
to budget cuts that put academic and research competitiveness at jeopardy. Although there was 
some resistance both from students and faculty members against the new method, the college has 
set a goal for having 75 % of all core undergraduate courses transitioned to the hybrid format [12]. 

Another example of hybrid learning in higher education comes from North Carolina State Uni¬ 
versity where three sections of an introductory-level engineering graphics course was taught as a 
hybrid course. Students were required to meet face-to-face once per week, and the online com¬ 
ponent of the course included voiced-over presentations, sketching demonstrations, and software 
application demonstrations. The class time was mainly used to discuss key solid modeling topics, 
check homework, and answer questions about assignments. Students were also required to take an 
online quiz before they attended the face-to-face session. The results of the study indicated that 
the final exam mean for the hybrid sections was slightly higher than face-to-face sections, but this 
difference was not statistically significant. However, the instructors indicated that meeting face-to- 
face once per week created chaos because they felt like they needed to cover content despite the 
fact everything was available online [13]. This dovetails very well with Peercy and Cramer’s comment 
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that faculty members need to encourage time on task by assigning grade weights to the online 
activities in which students are expected to engage [1]. 

Hybrid learning has also been implemented in medical education, which indicates its practices 
can easily be transferred to any discipline. Educators from Johns Hopkins School of Medicine con¬ 
verted a cardiovascular section of a first-year physiology course to a hybrid format, and required 
students to watch the lectures before class. The class time, in turn, was used for question and answer 
sessions, interactive simulations, clinical correlations, discussions related to the current trends in 
research, and student presentations. This format enabled the faculty members to spend more time 
discussing the content with students. Goldberg et al. concluded, based on the survey data they 
collected, that students enjoyed the delivery format, found the combination of online lectures with 
in-class discussions useful and effective for their learning, and indicated that they were more likely 
to retain the content they learned in this class [14]. 

To successfully convert a traditional face-to-face course to a hybrid format, several dimensions 
need to be taken into consideration. As Peercy and Cramer noted “successful hybrid teaching 
cannot be a mish-mash of traditional lecturing with some online content but rather a thoughtful 
re-design of course pedagogy, and meaningful interactions with students” [1]. Maximizing success 
in a hybrid learning environment requires reconceptualization of teaching, learning, and assessing. 
Therefore, to transform a course from a traditional face-to-face format to hybrid, an instructor must 
carefully examine the learning objectives; develop new online and face-to-face activities; utilize 
new types of assessment (e.g., real-life projects, presentations); integrate face-to-face and online 
learning activities; and interact with students in new ways [11]. This process requires a planned and 
well-supported approach that includes an instructional model based on a sound learning theory, 
high-quality faculty development, course development assistance, learner support, and ongoing 
formative and summative assessment [5]. 


DESCRIPTION OF THE INNOVATION 

In the hopes of improving learning while providing some flexibility both for students and instructors, 
a junior-level construction engineering course, entitled Construction Equipment and Heavy Construc¬ 
tion Methods, was converted from traditional face-to-face instruction to hybrid format starting in the 
summer of 2012. This is a three-credit, junior-level course that originally included two lecture hours 
and a calculation lab lasting two hours each week. In the traditional format, lecture hours were used 
to deliver content through lecture presentations and demonstrating the solution of example problems. 
The lab periods consisted of problem solving and group work. The objective of the course conversion 
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was to replace the face-to-face passive content of the lectures and example problem solutions with 
the online content, and use some of the time released for interactive activities. Since the lab periods 
already consisted of active learning exercises, little change was contemplated for that portion of the 
class except to revise some of the problems to make them more open ended and realistic. The instruc¬ 
tor was able to use the lecture hours for in-class activities that were similar to lab activities, The online 
portion ensured that students obtained the necessary background information to successfully complete 
the in-class activities. Other assessments included assignments (30%) (e.g., homework, presentations, 
online modules), quizzes over readings from the textbook (10%), and exams and a comprehensive final 
(60%). A complete course calendar can be found online at http://tinyurl.com/natxwm5 

Embracing the benefits of hybrid learning that has been identified in previous literature (e.g., 
flexibility, being able to pause and rewind lectures), this study contributes to the field by providing 
a unique approach to hybrid course design. In addition to the online lectures, typically included 
in hybrid courses, the course under investigation utilized the capabilities of an e-learning content 
management to create interactive exercises (i.e., online modules) where students could work on 
example problems in a scaffolded online learning environment. Additionally, the online lectures, 
modules, and face-to-face activities meaningfully complement each other and aim to improve stu¬ 
dent learning. A more detailed description of the course conversion process is described elsewhere 
[15]. This study reports the design, development, and evaluation of this hybrid course and provides 
practical recommendations for hybrid course development. 

Online Activities 

Online activities for this hybrid course involved lectures and modules. The instructor created 
online videos for the lectures that he would normally give in a traditional face-to-face classroom 
format. These online lectures introduced the course concepts (e.g., bulldozers, asphalt paving, pile 
driving, and concrete placement) through visually enhanced, relatively short videos by checking 
comprehension along the way. A weblink to an example online lecture is included in Figure 1. Our 
experience has also shown that it takes less time to cover the same amount of material online in 
comparison to a face-to-face lecture. The length of the videos were taken into consideration while 
deciding how much of the classroom time could be reduced. For example, if students spend an hour 
online (watching videos and taking quizzes), the amount of face-to-face class time was reduced 
accordingly. Also, these online activities were scheduled so that students could attend activities like 
career fairs, field trips, and related activities without sacrificing any learning time. 

After the instructor screen recorded a given topic, an instructional design assistant engaged in 
post-production. In the online course design literature, it is recommended that the videos should be 
around five minutes to keep students engaged in the material [16]. Bearing this in mind, the videos 
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Hydraulic Excavators 


Can also do: 


Lift things, like a 
piece of a utility pipe 


• Digging the 
trench 

• Liting the pipe 
into the trench 



► 03:52 


04:27 ..Hill X 


Figure 1. Example lecture video (http://www.screencast.eom/t/A9MCvuwlyhOG). 


were split into relatively short, manageable parts. Then the instructional design assistant edited the 
presentations by adding images, highlighting important information based on the principles of the 
cognitive theory of multimedia learning [17]. Finally, the assistant integrated the comprehension ques¬ 
tions checking students’ understanding of the material (See Figure 2). Students had two attempts 
for these questions and they received immediate feedback whether they got the right answer. These 
scores were recorded in the course management system (i.e. Blackboard Learn) and included as 
part of the overall course grade. In total, there were nine online lecture videos in this hybrid course. 

Online modules are interactive exercises that demonstrate how to solve problems in construction 
engineering and allow students to practice solving similar problems in a step-by-step, interactive 
scaffolded learning environment. Twelve modules were developed using the e-learning software 
Lectora Inspire® [18] for this hybrid course. These interactive exercises included various question 
types such as numerical entry, fill-in-the-blanks, drag and drop, and matching. All the modules 
met four instructional design decisions: 1) scaffolded problem solving, 2) step-by-step problem 
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_ QueStl0n 1 _I @ 20f27@ 

Instructions: Choose the correct statement(s) that apply Back Next 

Hydraulic Excavators 

— A Are e ffj c i en t because they can rotate around the central axis. 

□ B Can lift things. 

□ C Use hydraulic cylinders that can only push. 

□ D Use hydraulic cylinders to operate their boom, stick, and bucket. 


Check Answer 


Figure 2. Example comprehension question embedded within a lecture video. 


solving, 3) immediate explanatory feedback, and 4) multiple attempts. Figure 3 displays the screen- 
shot of an example online module, and a video explaining how the modules work can be found 
at http://www.screencast.eom/t/MznFBUMh6aF. 

To scaffold student learning in complex problem solving, worked-examples demonstrate how 
to solve a particular problem [19]. For this course, worked-examples were called “how-to videos” 
and students accessed them on needs-basis. The how-to videos demonstrate the solution steps 
and processes using different sets of numbers and data. If students had difficulty solving a given 
problem, they had the opportunity to watch a how-to video before they solve the graded problem. 

The online modules included a complex problem statement divided into multiple steps and 
students needed to submit their answers for each step. The underlying reason for taking students 
through the steps of a problem was to help them understand the whole process of solving a com¬ 
plex problem. Students were expected to implement this strategy in the open-ended problems that 
they solved during in-class sessions. Students also received automated explanatory feedback to 
ensure they knew what the right answer was and they did not carry mistakes forward. Students had 
two attempts in a given question to get the right answer, particularly for questions that required a 
numeric entry in order to avoid any calculation errors. 
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Figure 3. Example online module. 


Face-to-Face Activities 

The lecture-lab division in the course almost vanished in the hybrid format since the majority of 
the activities done during face-to-face meetings resembled lab activities. Taking the lecture and 
problem practice parts of the course to the online platform forced the instructional design team 
to reflect more on the face-to-face activities. Assignments were revised and converted into more 
open-ended, real-life exercises (chosen from construction projects on and around the campus) 
on which students can work in small groups under the supervision of the instructor and teaching 
assistants. For example, three assignments were converted to an ill-defined problem format by fol¬ 
lowing Jonassen’s instructional design model for ill-structured format [20]. In these activities, the 
instructor and the teaching assistants played a facilitative role rather than directing the instruction. 
Other in-class activities involved students presenting information, analyzing real-life scenarios and 
providing alternative solutions, and working on hands-on projects. 

Figure 4 displays an example in-class assignment. For this assignment, students received an 
instructions sheet, plans and blueprints of the jobsite, and a crane charts document. They worked 
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Miller Elementary Structure Erection 

Instructions: For this assignment, you will act as if you are the project manager constructing the new Miller 
Elementary. You will need to check if the crane owned by the company is sufficient to place the structural 
steel joists and precast concrete. A list of joist and concrete panel weights are provided on the following page, 
and plans and drawings are provided. 


You will need to determine 

• Location of all listed components. List plan sheet where the component is located. 

• The construction sequence— Include any major immediate steps such as the slab pour, construction 
of CMU bearing walls for joists, beam placement, etc. 

o Time, scheduling, cost etc. are not necessary. That class is next semester. Just the sequence 
is needed. 

• Boom length, boom angle, distance from face of building to the crane, radius at which the load will be 
placed, and approximate crane location (northeast of section, to the south of gymnasium etc.) 

o Joists are typically picked 3 at a time. However, this is not a requirement if the crane is near 
capacity. 

You may ask for any clarifications you feel you need to complete the assignment. Groups of 3 have been 
assigned. 


Screen shot of a page of the plan. 



Figure 4. Example In-Class Assignment. 
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in groups of three to four to design the whole crane sequencing process using the concepts they 
learned in online lectures and modules. The teaching assistant walked around the classroom to pro¬ 
vide assistance and guidance as needed. Students turned in their assignments at the end of the class. 


RESEARCH QUESTIONS 

The following research questions will be addressed in this study to evaluate the effectiveness of 
hybrid learning. 

1. Do students perform better in a hybrid learning environment compared to a traditional learning 
environment in a construction engineering context? 

2. What are student perspectives on the online component of a hybrid construction engineering 
course? 

a. What are student perspectives on the online lectures? 

b. What are student perspectives on the online modules? 

3. What are student perspectives on the face-to-face component of a hybrid construction engi¬ 
neering course? 

4. What are student perspectives on the overall hybrid course design in a construction engineer¬ 
ing course? 


RESEARCH METHODOLOGY 


The degree of student learning satisfaction plays an important role in evaluating the effective¬ 
ness of blended and hybrid learning environments [21]. A mixed-method approach was adopted in 
this study to investigate the impact of hybrid course design on student learning and satisfaction. A 
combination of quantitative and qualitative data enabled triangulating multiple sources of data to 
establish trustworthiness and consistency in interpretation [22]. 

Research Context 

The study was conducted in the Department of Civil, Construction, and Environmental Engineering 
at a large Land-Grant university in the Midwest of the USA. The department contains two academic 
programs—civil engineering and construction engineering—both nationally accredited by ABET. 
Approximately 1,000 students are enrolled in the department, about 20% of whom are female. The 
percentage of students enrolled in the construction engineering program makes up about 30% of 
the total enrollment. 
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Gender* Age** Classification*** 

Male Female Average Junior Senior Graduate 

47 6 21 41 11 1 

* 3 missing data **10 missing data *** 3 missing data 

Table 7. Participant information. 

Participants 

The summative evaluation included in this paper was collected in Spring 2014 and Fall 2014, in which 
19 and 48 students were enrolled, respectively. Out of these 67 students, 56 took an end-of-semester 
survey, and 13 students from Spring 2014 participated in focus group interviews (See Table 1). Pseudo 
names are used throughout this report to protect the identity of the participants. 

Data Sources and Analysis 

Quantitative data included student final grades and an end-of-semester survey. Qualitative data, 
on the other hand, included focus group interviews with students. The end-of-semester feedback 
survey was developed based on the benefits of hybrid learning identified in earlier research [5, 7,11], 
and the design of the particular course under investigation. The survey included four main sec¬ 
tions asking students’ perspectives on the effectiveness of the online component, the face-to-face 
component, the overall satisfaction with the hybrid course format, and demographic information 
on a five-point Likert scale. During analysis, Likert scale items were converted to numerical entries, 
1 being strongly disagree to 5 being strongly agree. The internal consistency reliability (coefficient 
alpha) of the survey was 0.82. Two open-ended questions asking what went well and did not go so 
well in the course were included to give students an opportunity to comment on issues that were 
not covered in the survey items. Descriptive statistics was used to analyze quantitative data; mainly, 
the percentages, mean scores, and standard deviations were included in the report. 

Focus group interviews were conducted to elaborate more on the issues raised in the surveys as 
well as to discuss the areas that were not covered through surveys. Fourteen students, divided into 
three groups, participated in the focus group interviews. All the interviews were administered during 
a class period in week 14 of a 15-week semester in Fall 2014 in the regular classroom where students 
had met throughout the semester. The first author of this paper, who assumed the instructional 
designer role, conducted the interviews with the help of a graduate student. Neither the instructional 
designer nor the graduate assistant were in direct contact with students during the semester, and 
neither had any impact on how students were graded. The study was reviewed and approved by 
the Institutional Review Board at the university. A semi-structured interview protocol was used, 
and included questions about the main aspects of the course and follow-up questions based on 
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Traditional 



Hybrid 



Grade 

Fall 2010 

N=43 

Spring 

2011 

N=43 

Fall 2011 

N=40 

Spring 2012 
N=29 

Fall 2012 

N=32 

Spring 2013 
N=34 

Fall 2013 

N=29 

Spring 

2014 

N=18 

Fall 2014 

N=43 

A 

16 

12 

19 

10 

23 

20 

12 

7 

14 

B 

14 

19 

14 

11 

4 

11 

12 

7 

24 

C 

9 

9 

7 

8 

5 

3 

3 

3 

5 

D or below 

4 

3 





2 

1 


Table 2 

. Grade distribution over time. 







participants’ answers. The interviews took an average of 30 minutes. All the focus group sessions 
were audio-recorded and transcribed verbatim for analysis. The interview transcripts were manually 
coded for recurring themes and categories following an open coding system. To ensure the validity 
of findings, qualitative data were triangulated by the quantitative survey data. 

RESULTS 


The findings from final course grades, the survey, and the focus group interviews will be pre¬ 
sented to answer the four research questions listed above. The first research question investigated 
whether hybrid learning was a more effective approach than traditional teaching approaches in a 
construction engineering context following a quasi-experimental procedure. 

Learning Outcomes 

The conversion of the course from a traditional format to a hybrid format started in Fall 2012. In 
order to answer the first research question, final course grades received in five semesters of hybrid 
format (Fall 2012 to Fall 2014) were compared to the ones in four semesters of traditional format. The 
same instructor taught both the formats, and identical assignments, exams, and assessments were 
included in the final grades except for the online activities in the hybrid version. Table 2 displays a 
detailed distribution of grades for each semester, and Figure 5 displays a graphical comparison of 
the grades received in traditional and hybrid formats. 

The results indicated that 49% of the students in the hybrid format received As while only 37% 
received As in the traditional format. The percentage of Bs seemed to be equal (37%) in both for¬ 
mats. However, the percentage of Cs dropped down to 12% in the hybrid format, while it was 21% 
in the traditional format. The percentage of students who received a D or below was only 1.92% in 
the hybrid format, while it was 4.52% in the traditional format. The results of the Kruskal-Wallis test 
also confirmed that this difference was statistically significant (H (1)=7.23, p< .01) (See Table 3). 
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Traditional N=155 Hybrid N=156 

(Fall 2010-Spring 2012) (Fall 2012-Fall 2014) 



■ D or below 

■ C 

■ B 

■ A 


Student Perspectives on Online Lectures 

The second research question addressed the student perspectives on the online component of the 
hybrid versions of the course, which included online lectures and modules. The results indicated that 61% 
of the students watched all the assigned lectures. Thirty percent of them indicated that they watched 
more than two-thirds (See Table 4). This was an improvement from the earlier versions of the course 
when the online activities were made available but not required, which confirmed the instructional de¬ 
sign team’s decision on making the online activities part of the student grade as a motivational factor. 

One predicted advantage of online lectures was the long-term availability for review, so we asked 
students if they watched any lectures more than once. Table 4 represents that 57% of the survey 
participants did not watch any of the videos more than once. The percentage of students who 
indicated having watched less than one-third of the lectures more than once was 39%. However, 
the focus group interviews indicated that students preferred pausing, rewinding, and re-watching 
portions of videos instead of re-watching the whole video. 


N DF 

ChiSquare 

Prob>ChiSq 

311 1 

7.2294 

0.0072* 

Table 3. Kruskal Wallis Test Results. 




12 


FALL 2016 











































ADVANCES IN ENGINEERING EDUCATION 

Evaluation of Hybrid Learning in a Construction Engineering Context: 
A Mixed-Method Approach 




none 

less than 
1/3 

between 

1/3 and 2/3 

more than 
2/3 

all 

I watched of the assigned online lectures for 

0% 

4% 

5% 

30% 

61% 

this class. (M=4.48, SD=0.76) 

(0) 

(2) 

(3) 

(17) 

(34) 

I watched of the lectures more than once. (M=l .50, 

57% 

39% 

2% 

0% 

2% 

SD=0.71) 

(32) 

(22) 

(1) 

(0) 

(1) 


Table 4. The amount of online lectures watched (N=56). 


Student perspectives on the effectiveness of online lectures were examined through four items 
(See Table 5). The majority of participants (58%) agreed or strongly agreed that videos were short 
enough to keep them engaged, while 21% disagreed with it. One student, Sam, in the focus group 
interviews noted that some lectures were longer than they needed to be because he was quite 
familiar with the topic as he stated, “I mean if you are not familiar at all then it was probably a great 
lecture but if not you get a little unstable” (Sam, Focus Group 3, p. 3). Other students commented 
that lectures were clear and easy enough to be covered in the online format. Likewise, 75% of stu¬ 
dents positively responded that online lectures contributed to their understanding of the material. 

A great majority of the survey participants (84%) agreed or strongly agreed that visual enhance¬ 
ments in lecture videos, which took considerable post-production time and effort, contributed to 
their understanding of the material. In order to keep students engaged in the material, the videos 
were split into short segments with embedded comprehension questions. Survey participants were 
asked if the comprehension questions measured their understanding of the material, and 57% re¬ 
sponded positively, 29% were neutral, and 15% responded negatively. Some students in the focus 
group interviews noted that questions were easy enough that they were able to answer them without 
actually watching the video. 



Strongly 

Disagree 

Disagree 

Neither Agree 
nor Disagree 

Agree 

Strongly 

Agree 

The videos were short enough to keep me engaged in the 

0% 

21% 

21% 

54% 

4% 

material. (M=3.39 SD=0.87) 

(0) 

(12) 

(12) 

(30) 

(2) 

Online lectures increased my overall understanding of the 

4% 

7% 

14% 

68% 

7% 

material for this course. (M=3.68 SD=0.86) 

(2 

(4) 

(8) 

(38) 

(4) 

The visual enhancement in lecture videos (e.g. pictures, 

2% 

5% 

9% 

66% 

18% 

graphics, text explanations, etc.) increased my overall 
understanding of the material for this course. (M=3.93 SD=0.81) 

(1) 

(3) 

(5) 

(37) 

(10) 

The comprehension questions embedded in the online 

4% 

11% 

29% 

55% 

2% 

lectures ensured that I understood the material for this 
course. (M=3.41 SD=0.85) 

(2) 

(6) 

(16) 

(31) 

(1) 


Table 5. Effectiveness of online lectures (N=56). 
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none 

less than 
1/3 

between 
1/3 and 
2/3 

more 

than 2/3 

all 

I watched 

of the assigned online modules for this class. 

0% 

2% 

7% 

21% 

70% 

(M=4.59, SD=0.71) 


(0) 

(1) 

(4) 

(12) 

(39) 

I watched 

of the how-to videos in the online modules 

5% 

30% 

39% 

18% 

7% 

(M=2.91, SD=1.00) 


(3) 

(17) 

(22) 

(10) 

(4) 


Table 6. The amount of online modules watched (N=56). 


Student Perspectives on Online Modules 

According to the survey results, 70% of the participants indicated that they did all the assigned 
online modules while only 2% did less than one-third of them. An important part of online modules 
was “how-to” videos, which were on-demand videos explaining the problem solution step-by-step 
with a different data set. As can be seen in Table 6, only 25% of the survey participants indicated 
that they watched all or more than two-thirds of the how-to videos. This was an expected result be¬ 
cause the goal of those how-to videos was to scaffold student learning in complex problem solving. 
If students were able to solve the problem on their own, they did not need to watch the additional 
videos. However, they were able to access these explanatory videos when they needed. 

Student perspectives on the effectiveness of online modules were examined through five items 
(See Table 7). A reasonably high percentage of participants (84%) responded positively that online 
modules increased their understanding of the material for this course, and 71% of the students posi¬ 
tively rated how-to videos in the online modules indicating that they increased their understanding 



Strongly 

Disagree 

Disagree 

Neither 
Disagree 
nor Agree 

Agree 

Strongly 

Agree 

Online modules increased my overall understanding of the 

0% 

4% 

13% 

64% 

20% 

material for this course. (M=4.00 SD=0.69) 

(0) 

(2) 

(7) 

(36) 

(11) 

How-to videos in online modules increased my understanding of 

2% 

7% 

20% 

50% 

21% 

the problem solution process in a given topic. (M=3.82 SD=0.92) 

(1) 

(4) 

(11) 

(28) 

(12) 

The step-by-step problem solution in online modules increased 

4% 

4% 

13% 

55% 

25% 

my overall understanding of the material for this course. 

(M=3.95 SD=0.92) 

(2) 

(2) 

(7) 

(31) 

(14) 

The feedback I received from the system during modules 

2% 

25% 

36% 

38% 

0% 

contributed to my understanding of the material for this course. 
(M=3.09 SD=0.84) 

(1) 

(14) 

(20) 

(21) 

(0) 

Knowing that I had two attempts to get the right answer for a 

2% 

18% 

25% 

43% 

13% 

question decreased my anxiety of making mistakes. (M=3.46 
SD=0.99) 

(1) 

(10) 

(14) 

(24) 

(7) 


Table 7. Effectiveness of online modules (N=56). 


14 


FALL 2016 















ADVANCES IN ENGINEERING EDUCATION 

Evaluation of Hybrid Learning in a Construction Engineering Context: 
A Mixed-Method Approach 



of the problem solution process. Similarly, focus group interview findings indicated that students 
found how-to videos as helpful features of the online modules. For example, John mentioned that 
he liked the “how to” videos the best because he could go back and watch the associated “how-to” 
video when he had difficulty understanding a concept or a problem. For another example, Brad 
commented: 

Most of the modules and the videos that went along with them and/or click on the inside 
notes, the videos actually went step by step. If you could not figure out on your own, lots of 
videos he [the instructor] went through [the problem]. (Brad, Focus Group 1, p. 5) 

Step-by-step problem solving in modules was positively rated by 80% of the survey participants. 
Focus group findings also confirmed this finding that students referred to this feature when they 
were asked what they liked about the online activities. The step-by-step procedure not only taught 
students the steps to solve the problem, but also reduced student stress because when they made 
a mistake, it was easier to identify the error and that error was not carried forward to the next levels 
of the problem. For example, Bruce noted: 

If you are off by a certain percentage or I mean if you have one calculation wrong it didn’t 
screw up your whole problem. You didn’t miss the problem compared to maybe partial 
credit in class if you did [it] hand-written. (Bruce, Focus Group 2, p. 2) 

Students also received automated feedback based on their answers. If they could not give the 
right answer even in the second attempt, the system provided the correct answer with a brief expla¬ 
nation. Although students liked the immediacy of this type of feedback, technical issues they had 
with the e-learning system hindered their learning at times. That might be the reason why 27% of 
the survey participants rated the item on feedback negatively. In particular, questions that required 
a numeric entry had some rounding errors, so students were told that their answer was wrong even 
though they did everything right but did not round the numbers as the system expected. Therefore, 
the feedback was not really helpful for them as Sam noted: 

One time they would round one way, the next round the other. I mean you just had to try to 
outguess it. That was frustrating. The thing when you are doing something online, the feedback 
is great. That way you know if you did wrong or right. But then when you get to that point where 
it’s telling you [you are wrong]. It interrupts actual learning because now you are trying to figure 
out what you did wrong while you didn’t do anything wrong. (Sam, Focus Group 3, p. 2) 
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Strongly 

Disagree 

Disagree 

Neither 
Disagree 
nor Agree 

Agree 

Strongly 

Agree 

We had more time in class to solve engineering 

0% 

7% 

20% 

71% 

2% 

problems because we watched the lectures outside 
the class. (M=3.68 SD=0.64) 

(0) 

(4) 

(11) 

(40) 

(1) 

The class time was more beneficial to my learning 

5% 

11% 

27% 

48% 

9% 

than a typical lecture format. (M=3.45 SD=0.99) 

(3) 

(6) 

(15) 

(27) 

(5) 

The activities we did in the face-to-face sessions 

0% 

7% 

16% 

63% 

14% 

were applicable to my future career. (M=3.84 
SD=0.76) 

(0) 

(4) 

(9) 

(35) 

(8) 

I liked being able to work in small groups in the 

2% 

2% 

4% 

54% 

39% 

class. (M=4.27 SD=0.77) 

(1) 

(1) 

(2) 

(30) 

(22) 

Working in small groups increased my overall 

2% 

7% 

5% 

50% 

36% 

understanding of the material for this course. 

(M=4.11 SD=0.93) 

(1) 

(4) 

(3) 

(28) 

(20) 

I had sufficient opportunity to ask for further 

0% 

0% 

4% 

45% 

52% 

clarification to the instructor and TAs. (M=4.48 
SD=0.57) 

(0) 

(0) 

(2) 

(25) 

(29) 


Table 8. Effectiveness of face-to-face activities (N=56). 


The last item in the online modules asked students if having multiple attempts reduced their 
anxiety of making mistakes. The results indicated that responses were scattered throughout the 
agreement scale. Students participating in focus group interviews noted that having three attempts 
might be more helpful particularly with questions requiring a numeric answer. They indicated 
that the system was inconsistent in whether to round the numbers or not, so they were getting 
incorrect answers even though they ensured everything was correct. Therefore, the number of 
attempts will be increased to three for questions that require a numeric answer in the future 
offerings of the course. 

Student Perspectives on Face-to-Face Activities 

The third research question addresses the students’ perspectives on the face-to-face component 
of the hybrid course design, which were summarized in Table 8. It was important to see if students 
realized the value of creating opportunities for solving real-life problems, which was confirmed by 
73% of the survey participants. They positively rated that the online activities released extra time 
for solving complex engineering problems. Student responses were more scattered for whether the 
class time was more beneficial to their learning than a typical lecture format. Class time was used 
more effectively than a traditional lecture format according to 57%, while 16% disagreed with it. One 
student in focus group interviews mentioned that he preferred the traditional lecture format better 
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than any other online format because he wanted someone to teach him rather than him trying to 
teach himself as can be seen in the following quote: 

I think, I am more of a fan of regular lectures. I have never had any problems sitting through 
them like for 50 minutes or whatever. I just feel like I pay a lot of money to go to school 
and have someone teach me, not being able to teach myself and then work in groups. But 
I understand that I’d have to do that sometimes even in regular lectures. (Allen, Focus 
Group 1, p. 1) 

Another student, on the other hand, noted that he was excited when he heard that this class was 
hybrid because he liked the flexibility that online and hybrid formats could bring, and he even recom¬ 
mended that this class could even be completely online (Brad, Focus Group 1, p. 1 & 9). Such contrasting 
views are typical because in a group of students this size, it is likely that there will be students who prefer 
traditional teaching approaches to other student-centered approaches. Focusing on whether students 
learn, rather than who teaches it, is more likely to be useful. In-class activities were applicable to stu¬ 
dents’ future career according to 77% of the survey participants. Focus group findings also confirmed 
this finding that students enjoyed the real-life tasks. For example, a construction professional brought 
in projects that he himself worked and asked students how they would perform the tasks of the proj¬ 
ect. Students repeatedly conveyed that they enjoyed the activity, and the feedback that speaker gave 
during the activity forced them to look at the issue from multiple perspectives. Ray noted: 

And so he passed us [plans and drawings], where would you place your crane; how would 
you go about building, the school? And setting all the stuff. It was good, he came around 
and before we finished he asked us, OK so what do you guys have so far? What do you 
guys think? You would tell him and then he would throw something in whether you know 
something you didn’t think about previously so then you need to go back and re-think 
about it. It was just good having his input right there. (Ray, Focus Group 1, p. 7) 

Other in-class activities were also designed as open-ended, ill-defined, and realistic problems 
so that students could immerse themselves in the professional environment as much as possible. 
John commented: 

I do like the in-class assignments, how they put like real-life situations, like we had to do the 
footings or dig out the basement for [a building on campus]. I did enjoy the live perspective 
of it. So I thought it was good. (John, Focus Group 2, p. 5) 
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Strongly 

Disagree 

Disagree 

Neither 
Disagree 
nor Agree 

Agree 

Strongly 

Agree 

I feel like I will be able to retain what I learned in this class. 

2% 

13% 

21% 

59% 

5% 

(M=3.54 SD=0.85) 

(1) 

(7) 

(12) 

(33) 

(3) 

I would like to see this type of hybrid teaching in more of my 

12% 

18% 

35% 

29% 

6% 

courses. (M=3.02 SD=1.12) 

(6) 

(13) 

(14) 

(20) 

(3) 

I would recommend taking hybrid courses to my 

11% 

11% 

36% 

39% 

4% 

friends. (M=3.14 SD=1.03) 

(6) 

(6) 

(20) 

(22) 

(2) 


Table 9. Overall course satisfaction (N=S6). 


Working in small groups on open-ended problems was a major component of in-class activities, 
which was positively rated by 93% of the survey participants, and 86% indicated that working in 
groups contributed to their understanding of the material. Focus group findings also confirmed that 
group work was a favorable activity for students in this class. For example, Ray noted: 

What I liked about that was, in class, if you work in a group that is good, because you can, 

[and] if somebody doesn’t know it in your group, you have the ability to teach that person 
and that just means that you really understand that; you know what is going on. Plus you 
can obviously pull from each person so you can combine all the information. I may not but 
there is another person will, I can teach somebody else. With this class, because we have 
been together for [a] couple semesters now, we got to know each other, [and] it works out 
really well. (Ray, Focus Group 1, p. 7-8) 

Finally, students were asked whether they had enough opportunities to interact with the instructor 
and teaching assistants. Because the teaching staff did not need to spend class time giving lectures, 
it was our hope that they would be able to provide more guidance to students on an individual 
or group basis, which was positively rated by 97% of the survey participants. These participants 
indicated that they had enough opportunities to ask questions for further clarification from the 
teaching staff. Focus group findings also confirmed that students indicated they learned the most 
by interacting with teaching assistants. 

Student Perspectives on Overall Hybrid Course Design 

The last research question investigates the student perspectives on overall hybrid course design 
model. Overall course satisfaction, student-identified benefits, and challenges will be reported in 
this section (See Table 9). The majority of the participants (64%) positively rated that they would be 
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able to retain what they learned in the class; however, only 35% wanted to see more hybrid courses, 
and 43% recommended hybrid courses. 

Two major concerns were raised in previous studies on hybrid instruction: disjoint between online 
and face-to-face activities and the “course and a half syndrome”—adding extra online activities on 
top of regular traditional classroom activities [11]. In the hybrid course under investigation here, the 
online and in-class activities were carefully tailored so that online activities would prepare students 
for the in-class activities in order to avoid any feelings of disconnect. Additionally, the online activi¬ 
ties were regarded as an important part of the learning process, and face-to-face meeting time was 
reduced based on how much time students spent doing online activities outside of class watching 
lecture videos and working on modules. The workload was just as manageable for the course ac¬ 
cording to 88% of the survey participants, which was also confirmed in the focus group interviews. 

The focus group findings indicated that online activities prepared students for the in-class activities, 
and they performed better in class if they did the online activities on time. For example, Jason noted: 
“They [online activities] did help a lot. I mean if you didn’t know what you are doing on the modules and 
then coming to class and doing the homework you are just as confused.” (Jason, Focus Group 2, p. 2) 
Some students also developed some study skills to make sure they were prepared when they 
meet with their peers and instructors in the face-to-face session. They took notes while doing the 
online activities even though it was not required. Brad explained: 

See, I went through and I wrote down each question and I mean 3-5 pages per module. I 
wrote down every step I am doing. When I got in class, the class assignment is like dum 
dum dum done, I am out. Because I know how to do it. I am not struggling through it. I was 
not flipping through my notes trying to find an example. (Brad, Focus Group 1, p. 5-6) 

Another student said that he did not take notes during the online activities because he was trying 
to get them done as quickly as possible. However, he finished Brad’s comment saying, “Then I was 
[flipping through notes]” (Allen, Focus Group 1, p. 6). The following quote by Brad summarizes how 
all the activities in a class complemented each other particularly from his point of view. 

Homework is such a small percentage of your grade. What is it, 10%? Who cares if you did 
them all right, you know. It’s, if you do homework just to get it done, then you are just, the 
rest of 90% of your grade is gonna suck. If you do homework to understand what the hell 
you are talking about and what it all means, exams are gonna be easy, projects are gonna 
be easy, everything is gonna be easy. The rest of your 90% is gonna be “wow.” (Brad, Focus 
Group 1, p. 6) 
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Student-Identified Benefits of Hybrid Learning 

Focus group interview findings pointed out that hybrid instruction provided two major benefits for 
students. Students repeatedly referred to the flexibility of the hybrid format as an advantage. They indi¬ 
cated that they enjoyed being able to do the online activities on their own time. For example, one student 
noted, “It’s nice to be able to do it when you have time for it” (Jane, Focus Group 3, p. 1). Another student 
pointed the opportunities for not being restricted to classroom time as pointed out in this comment: 

I liked hybrid learning, as far as you only have to come in to class [a] certain amount of 
days and then the rest you do it in your own time, which is really good. It just adds a lot of 
flexibilities into my schedule, which is the biggest thing, so a lot of open times. And I can do 
it either early in the morning or late at night and I don’t really have to worry about [it] with 
everything else I have going on. (Ray, Focus Group 1, p. 1) 

Another flexibility that the hybrid format brought was to create a context where students were able 
to plan and work ahead if they preferred to do so. All the online lectures and modules in this course 
were made available from the beginning of the semester, but they were closed after the due dates. 
Students appreciated this flexibility because they could work ahead if they knew they had a busy 
week coming up as Brad said: “If I have an hour gap, I’d like to fill it with something. And the ability to 
sit down and work something like this, I feel like, yeah, just knock it out.” (Brad, Focus Group 1, p. 2) 
Another advantage of hybrid format was the ability to pause and rewind the lecture videos. Students 
repeatedly referred to being able to pause, rewind, and replay as one positive aspect of hybrid learn¬ 
ing. For example, Sam mentioned that he was able to go back and review the material when he did not 
understand it or skipped parts of the video if he was confident in his knowledge in that area. Sam noted: 

[I liked] being able to pause it. If you need to replay it, if you didn’t get it all the first time, you 
can replay it and go back over it whereas other stuff you can also click ahead if you already 
know everything. So, it gives you that option. Kind of go in your pace. (Sam, Focus Group 3, p. 2) 

The online component also provided opportunities for students to run the lecture videos while 
they are working on their homework assignments so that they could practice what they learned im¬ 
mediately. For instance, Brad mentioned how having videos available helped him do the homework 
assignments. 

While you are working on the module, [I liked] being able to have the video up while you 
are working on the video at the same time. It will play along and go step by step to keep up 
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instead of frantically writing a bunch of notes while you are sitting in the class. (Brad, Focus 
Group 1, p. 6) 

Student-Identified Challenges of Hybrid Learning 

The analysis of focus group interviews revealed three main challenges in this hybrid course: 
technical issues, course organization, and communication issues. Although the instructional design 
team tried to do their best to test and debug all the online activities, they were unable to avoid some 
technical glitches, which hindered student learning at times as Brad pointed out: 

The frustrating part was when you go through all that, spending 15 minutes on the problem 
realizing that you had the right answer all along and then the computer was wrong...I just 
wanted to throw a chair [at] the computer. So yeah, it is just a lot of weird things in the 
system. (Brad, Focus Group 1, p. 4) 

Another challenge for students in this hybrid course was related to course organization. The way 
the syllabus was designed was not consistent in terms of when things were due. In other words, 
there was not a specific day of the week that students would do an online activity and another day 
for in-class activities, which created confusion at times. For example, Bruce noted: 

One of the things I found frustrating was coming to class and thinking that I had my 
homework done because I had it in my hand. And then oh, we had a module due last night. 

Just having to be aware of the, there is in-class and out-of-class, I didn’t care for that 
[others approving]. That got kind of confusing sometimes. (Bruce, Focus Group 2, p. 6) 

One final challenge identified by students was communication issues. Because students missed the 
opportunity of asking questions while watching the lectures and modules, they needed to compen¬ 
sate for that via online communication. However, there was not a set agreement between students 
and the teaching staff about how soon emailed questions would be answered, which sometimes 
resulted in delayed communication. 


DISCUSSION AND CONCLUSIONS 


Several findings are worth noting that can shed a light on hybrid course design. The discussion 
will also follow the order of research questions. Thus, we will discuss learning outcomes and student 
perspectives on online activities, face-to-face activities, and overall hybrid course design. 
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Our quasi-experimental design results indicated that students in the hybrid format performed 
at a significantly higher level than students in the traditional format, which confirmed findings from 
previous research [6, 7]. The fact that the number of students receiving As increased in the hybrid 
format indicated that students who would receive Bs in a traditional approach benefit from the 
flexibility and pacing features of hybrid course design by watching and re-watching the lecture 
material as they needed. Informal conversations with other instructors using a hybrid approach in 
their teaching also supported this hypothesis that students who are at a B range benefit the most 
from this approach, as they are motivated learners, but may not be able to keep up with the pace in 
a traditional approach. The hybrid format enables them to enhance their learning through pausing 
and re-watching. 

The majority of the students indicated that they did almost all the required online activities, and 
they opined that the workload was reasonable. This implied that the hybrid course under investiga¬ 
tion was successful in motivating students and did not succumb to the “course and a half syndrome” 
by overloading students with additional online activities on top of existing in-class activities [11]. 
The balance between the online and face-to-face activities was manageable. 

The fact that students could learn the content through online lectures and videos, which prepared 
them for more open-ended problem solving activities, indicated that the hybrid format was a good fit 
for this particular course. Positive ratings for online modules, in particular, implied that students enjoyed 
the interactive, step-by-step problem solving exercises, which were scaffolded through worked-examples 
and explanatory videos. E-learning tools provide opportunities for individual students to practice prob¬ 
lem solving in a safe environment where they can access resources based on their individual needs, 
which, in turn, reduces task complexity, provides structures, and reduces learner frustration [23]. 

Additionally, by taking the passive content out in the online platform, instructors could spend the 
precious in-class time engaging students in real-life engineering problems where they can work in 
groups. Positive ratings and comments about the effectiveness of working in small group confirms 
the previous findings that collaboration and cooperation work for promoting a range of learning 
outcomes by enhancing academic achievement, student attitudes, as well as interpersonal skills [24]. 

Finally, successful weaving of online and face-to-face activities helped students directly apply 
what they have learned online to in-class activities. Therefore, the course was a “whole” rather than 
a disjointed two parallel courses as recommended by Kaleta, Skibba and Joosten [11]. However, the 
research findings also indicated that not all students realized that the online part was an important 
feature of the course, and it prepared students for in-class activities and exams. This implied that 
students who lacked self-regulation and metacognitive strategies needed explicit instruction on how 
to be successful in hybrid courses [23]. The learning strategies that were adopted by some students 
that proved to be effective could be turned into required activities so that students who lacked those 
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skills can utilize the techniques and perform as well as their peers. For example, a few students men¬ 
tioned in the focus group interviews that they were taking notes during the online activities to be 
used as a reference in face-to-face activities. This can easily be implemented as a required worksheet 
activity that students fill in during the online activities. Monitoring student learning and providing 
such recommendations can also be another strategy to promote metacognitive strategies in students. 

Students referred to two main benefits of online learning: flexibility and being able to pause 
and rewind the videos. These two advantages of hybrid course design are repeatedly mentioned in 
the literature explaining the hybrid learning approaches. However, it was important in this study to 
provide the evidence from students’ perspectives that they realized these advantages. A majority of 
the students enjoyed being able to work on the course at a time that worked for them, which added 
flexibility to their schedules. Previous research findings also confirmed this finding that hybrid learn¬ 
ing offers flexibility for over-burdened students and for those with family and job responsibilities [5, 
11, 25]. Additionally, students could play, pause, replay, and review the lecture videos as opposed to 
the one-shot feature of a traditional lecture format. Students commented that they did not have to 
take notes as frantically as they would do in a typical lecture because they could control the videos, 
and they feel empowered rather than being overwhelmed [23]. 

Despite all the positive ratings and comments on the hybrid course design, students encountered 
a few issues, which hindered their learning at times including technical issues, course organization 
issues, and communication issues. Technical issues in the online activities, in particular with auto¬ 
mated feedback, not only created discomfort and frustration but also affected students’ learning 
experiences. In an online or hybrid course, technology should be seamless so that the focus can be 
on learning [11]. All the technical glitches need to be reduced to a minimum if not eliminated com¬ 
pletely. Complaints about the course organization issues also proved that hybrid courses need to be 
very carefully planned and implemented in a consistent manner [5]. Finally, students’ expectations 
of how to communicate with the instructors also change because they miss the opportunity of ask¬ 
ing questions during lectures. Instructors need to establish clear expectations of communication 
(e.g., return rate, communication channel) and make themselves available to communicate via online 
tools (e.g., chat, video-conferencing). 

Overall, the findings of this study indicate that hybrid learning has the potential to transform engi¬ 
neering education by creating “space” in face-to-face meeting times for more open-ended, realistic 
problems that can be worked on in small groups by adding advantages like flexibility and learner pac¬ 
ing. However, hybrid course design requires a careful reconsideration of learning objectives, learning 
activities, assessments, as well as communication channels. Based on the findings of this study, we 
can make the following practical recommendations for those who are interested in converting their 
traditional face-to-face classroom to a hybrid format: (1) make the online component required; (2) 
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provide scaffolding for online activities; (3) keep the number of technical issues to a minimum; (4) 
meaningfully weave the online and face-to-face activities; (5) prepare students for the new format; (6) 
provide a well-established and consistent structure; and (7) communicate with students in new ways. 

Limitations and Recommendations for Future Work 

Like most studies, this study encountered some limitations that should be taken into consideration 
when interpreting the results. First, the findings from this study mainly rely on self-reported data, 
which may at times be biased. The researchers made every effort to create a safe environment for 
participants to encourage sharing their honest opinions. Also, findings are triangulated using dif¬ 
ferent data sources (survey, student grades, and focus group interviews) to compare responses in 
each source of data. Second, a true experimental design would be ideal to answer the first research 
question. However, it was important that all the students participate in the hybrid format, so random 
assignment of students in two different approaches (traditional versus hybrid) was not feasible for 
this study. Also, when the comparison is done, student characteristics such as gender, student level, 
admission test scores were not controlled, as we did not have access to such information. On a related 
note, the grading scale in the hybrid version was slightly different than the one in the traditional 
version, as the online modules were added to the graded assignments, which might have influenced 
the results. However, as students received credit based on their performance on the lecture quizzes 
and online modules (as opposed to receiving credit for merely watching videos), they still needed to 
make an effort, and show evidence for learning the material. Finally, because the hybrid format was 
implemented in this particular course, we had to limit the participant pool to the students who were 
enrolled in the course. However, four more courses are currently in the process of conversion from 
traditional a face-to-face format to hybrid instruction. A larger-scale study involving all the hybrid 
courses might give a more generalizable data set informing the effectiveness of hybrid instruction. 
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